Learning Model Bias

In this paper the problem of learning appropriate domain-specific bias is addressed. It is shown that this can be achieved by learning many related tasks from the same domain, and a theorem is given bounding the number tasks that must be learnt. A corollary of the theorem is that if the tasks are known to possess a common internal representation or preprocessing then the number of examples required per task for good generalisation when learning n tasks simultaneously scales like O(a + ~), where O(a) is a bound on the minimum number of examples requred to learn a single task, and O( a + b) is a bound on the number of examples required to learn each task independently. An experiment providing strong qualitative support for the theoretical results is reported.